COMMENT ⊗   VALID 00004 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002	\input memo.tex[let,jmc]
C00010 00003	\smallskip\centerline{Copyright \copyright\ \number\year\ by John McCarthy}
C00011 00004	Notes:
C00017 ENDMK
C⊗;
\input memo.tex[let,jmc]
%smolen[e87,jmc]		On Smolensky's "Proper Treatment of Connectionism"

\title{Epistemological Challenges for Connectionism}

1. The notion that there is a subsymbolic level of cognition between a
symbolic level and the neural level is plausible enough to be worth
exploring.  Even more worth exploring is Smolensky's further conjecture
that the symbolic level is not self-sufficient, especially where intuition
plays an important role, and the causes of some symbolic events must be
explained at some subsymbolic level.  The possibility that present day
connectionism models this subsymbolic level is also worth exploring,
but I find it somewhat implausible.

	An example of Smolensky's proposal is that the content of some new
idea may be interpretable symbolically, but how it came to be thought of
may require a subsymbolic explanation.  A further conjecture, not explicit
in the paper, is that an AI system capable of coming up with new ideas may
require a subsymbolic level.  My own work explores the contrary conjecture
--- that even creativity is programmable at the symbolic level.
Smolensky doesn't argue for the connectionist conjectures in the paper,
and I won't argue for the logic version of the ``physical symbol system
hypothesis'' in this commentary.  I'll merely state some aspects of it.


2. The paper looks at the symbolic level from a certain distance
that does not make certain distinctions --- most important being
the distinction between programs and propositions and the different
varieties of proposition.

3. My challenges to connectionism entirely concern epistemology ---
not heuristics.  Thus I will be concerned with what the system finally
learns --- not how it learns it.  In particular I will be concerned with
what I call {\it elaboration tolerance}, which I call the ability of a
representation to be elaborated to take into account additional phenomena.

	From this point of view, the connectionist examples
I have seen suffer from what might be called the unary or even propositional
fixation of 1950s pattern recognition.   The basic predicates are all
unary and are even applied to a fixed object, and a concept is
a propositional function of these predicates.  The room classification
problem solved by Rumelhart, Smolensky, McClellan and Hinton (1986)
is based on unary predicates of rooms, e.g. whether a room contains
a stove.  However, suppose we would like the system to learn that the
butler's pantry is the room between the kitchen and the dining room
or that a small room adjoining only a bedroom and without windows is
a closet.  As far as I can see the RSMH system is not ``elaboration
tolerant'' in this direction, because its inputs are all unary
predicates on single rooms.  To handle the butler's pantry, one
might have to build an entirely different connectionist network,
with the RSMH network having no salvage value.
My epistemological concerns might be satisfied by an explanation of
of what the inputs and outputs would be for a connectionist network
that could identify all the rooms of a house, including those whose
identification depends on their relation to other rooms.

I might remark that the 1960s vision projects at Stanford and M.I.T. were
partly motivated by a desire to get away from the unary bias of the 1950s.
The slogan was ``description, not mere discrimination''.  Indeed one of
the motivations for starting on robotics was to illustrate and explore the
fact that to pick up a connecting rod a robot requires more than just
identifying the scene as containing a connecting rod; it requires a
description of the rod and its location and orientation.  Perhaps
connectionist models can do this, and it seems to me very likely that it
can be done subsymbolically.  I hope that Smolensky will address this
question in his response to the commentaries.

A semi-heuristic question of elaboration tolerance
 arises in connection with NETTALK, described
in (Rosenberg and Sejnowski 1987).  After considerable training the
network adjusts its 20,000 weights to translate written English into
speech.  One might suppose that a human's ability to speak is similarly
represented by a large number synaptic strengths learned over years.
However, an English speaking human can be told that in the roman
alphabet transcription of Chinese adopted in the PRC, the letter Q
stands for the sound |ch|, and the letter X for the sound |sh|.
He can immediately use this fact in reading aloud an English text
with Chinese proper names.  Clearly this isn't accomplished by
instantly adjusting thousands of synaptic connections.  It would
be interesting to know the proper connectionist treatment of how
to make systems like NETTALK elaboration tolerant in this way.
\smallskip\centerline{Copyright \copyright\ \number\year\ by John McCarthy}
\smallskip\noindent{This draft of \jobname\ TEXed on \jmcdate\ at \theTime}
\vfill\eject\end
Notes:

pandemonium, broadcast
combination of facts
why does AI need connectionism
q in Chinese
misprint on p. 19 l. -8 : contrued → construed
need more than modus ponens
How does it get a goal?  It looks like a lot of human setup of
the problem has been required so far.
Boltzmannize circumscription
p.27 - The system's knowledge of Ohm's law is distributed ...
Smolensky's performance-competence distinction doesn't seem to
correspond to that of the linguist.  Rather two levels of performance
description seem to be in question.

The effect of blood chemistry on ideas.
This would be worth a formal psychological experiment.  Matched groups
are asked to discuss the same topic before and after a fatiguing
experience or before and after drinking or taking various drugs.